Computational Psychiatry

Risky decisions involve choosing between options where the outcomes are uncertain. Cognitive tasks such as the CANTAB Cambridge Gamble Task (CGT) have revealed that patients with depression make more conservative decisions, but the mechanisms of choice evaluation underlying such decisions, and how they lead to the observed differences in depression, remain unknown. To test this, we used a computational modelling approach in a broad general-population sample (N = 753) who performed the CANTAB CGT and completed questionnaires assessing symptoms of mental illness, including depression. We fit five different computational models to the data, including two novel ones, and found that a novel model that uses an inverse power function in the loss domain (contrary to standard Prospect Theory accounts), and is influenced by the probabilities but not the magnitudes of different outcomes, captures the characteristics of our dataset very well. Surprisingly, model parameters were not significantly associated with any mental health questionnaire scores, including depression scales; but they were related to demographic variables, particularly age, with stronger associations than typical model-agnostic task measures. This study showcases a new methodology to analyse data from CANTAB CGT, describes a noteworthy null finding with respect to mental health symptoms, and demonstrates the added precision that a computational approach can offer.

Validated instruments such as questionnaires, patient-reported outcome measures and clinician-rated psychopathology scales, are indispensable for measuring symptom burden and mental state, and for defining outcomes in both psychiatric practice and clinical trials. Most often, the values on the instruments multiple items (dimensions) are added to derive a single, univariate (scalar) sum-score. Although this approach simplifies interpretation, there are always many possible combinations of individual items that can yield the same sum-score. Two patients can therefore obtain identical scores on a given instrument, despite having very different combinations of underlying item scores corresponding to different patterns of clinical symptoms. The same is also true when a single patient is measured at two different time points, where the resulting sum-scores can obscure changes that may be clinically meaningful. We present an alternative analytic framework, which leverages geometric concepts to represent measurements as points in a vector space. Using this framework, we show why sum-scores obscure information present in measurements of clinical state, and also provide a straightforward algorithm to mitigate against this problem. Clinically-relevant outcomes, such as remission or patient-centered treatment goals, can be represented intuitively, as reference points or anchors within this space. Using real-world data, we then demonstrate how measuring the relative distance between points and anchors preserves more information, allowing outcomes such as proximity to remission, to be defined and measured.

Most investigators in precision psychiatry force models to predict clinically meaningful but ultimately predefined outcomes in a high-risk population. We instead advocate for an alternative approach: let the data reveal which symptoms are predictable with high accuracy and then assess whether those predictable symptoms warrant early intervention. We correspondingly introduce the Sparse Canonical Outcome REgression (SCORE) algorithm, which combines items from clinical rating scales into severity scores that maximize predictability across time. Our findings show that this simple shift in perspective significantly boosts prognostic accuracy, uncovering predictable symptom profiles such as social difficulties and stress-paranoia from those at clinical high risk for psychosis, and social passivity from infants at genetic high risk for autism. The predictable scores differ markedly from conventional clinical metrics and offer clinicians memorable, actionable insights even when full diagnostic criteria are unmet.

BackgroundPavlovian responding is a core component of behavior and can be measured via Pavlovian-instrumental transfer (PIT), where Pavlovian responses bias instrumental actions. Standard single-lever PIT paradigms, which assess responses using a single-choice option, cannot dissociate the contribution of model-free versus model-based reinforcement learning. While indirect evidence suggests a role for model-free responding in single-lever PIT, the contribution of model-based strategies is unclear. It also remains unknown whether internal cognitive states, such as mind wandering, impair specifically model-based but not model-free PIT, as is theoretically expected. MethodsWe developed a novel, trial-by-trial two-stage PIT paradigm designed to computationally dissociate model-free and model-based Pavlovian responding by leveraging probabilistic state transitions and trial-wise outcome predictions. After each two-stage Pavlovian learning trial, participants performed a single-lever PIT trial as well as a query trial of explicit value judgment. Detailed task instructions were provided to support potential model-based strategies. Computational modeling was used to quantify individual learning strategies. We assessed mind-wandering questionnaires and thought probes. ResultsAnalysis of query and PIT trials revealed trial-by-trial updating of outcome expectations based on probabilistic task structure, consistent with model-based Pavlovian responding. Behavioral responses during PIT were best explained by a computational model-based reinforcement learning model. In contrast, we found little evidence for model-free Pavlovian responding. Higher levels of mind wandering were associated with reduced model-based control but did not impact model-free indices. ConclusionWe introduce a novel single-lever PIT paradigm that enables fine-grained dissociation of model-free versus model-based Pavlovian response systems. Our findings provide evidence that single-lever PIT can operate through model-based mechanisms, challenging the assumption that single-lever PIT is predominantly model-free. Our findings also indicate that internal attentional states selectively modulate model-based PIT. Given the involvement of Pavlovian responding in numerous psychiatric conditions, our paradigm offers new avenues for understanding maladaptive behavior. Author SummaryOur daily actions are often influenced by cues like the smell of food or the sound of phone notifications that signal potential rewards or losses. These Pavlovian cues can shape our instrumental behavior even though their outcomes do not depend on what we do - a process known as Pavlovian-instrumental transfer (PIT). Here we study the computational learning mechanisms that underlie such PIT effects. While it is often assumed that Pavlovian responding follows simple, automatic rules without a cognitive model of cue consequences (i.e., model-free), evidence also shows a role for cognitive anticipations in Pavlovian responding (i.e., model-based). In this study, we extend this evidence by showing that PIT responding can be driven by flexible model-based learning. We designed a task to test whether participants use model-free versus model-based strategies to guide PIT, providing detailed task instructions. Using reinforcement learning models, we found that most participants used model-based learning when forming cue-outcome associations. Importantly, peoples attention mattered: when they were more distracted and doing mind wandering, they relied less on model-based strategies. Our findings suggest that Pavlovian learning is complex, flexible, and influenced by internal mental states, opening new windows to understand decision-making problems in mental health conditions like addiction.

Maladaptive belief updating is a hallmark of psychiatric disorders, yet its underlying neurocomputational mechanisms remain poorly understood. While Bayesian models characterize belief updating in decision-making, they do not explicitly model neural computations or neuro-modulatory influences. To address this, we developed a recurrent neural network-based reinforcement learning framework to investigate decision-making deficits in psychiatric conditions, using schizophrenia as a test case. Agents were trained on a predictive inference task commonly used to assess cognitive deficits found in schizophrenia, including under-updating beliefs in volatile environments and over-updating beliefs in response to uninformative cues. The task thus included two conditions: (1) a change-point condition requiring adaptation in a volatile environment and (2) an oddball condition requiring resistance to outliers. We modeled these deficits by systematically manipulating key hyper-parameters associated with specific neural theories: reward prediction error (RPE) discounting and scaling (reflecting diminished dopamine responses), network dynamics disruption (reflecting impaired working memory), and rollout buffer size reduction (reflecting decreased episodic memory capacity). These manipulations reproduced schizophrenia-like decision-making impairments and revealed that suboptimal agents exhibited fewer unstable fixed points near network activity in the changepoint condition, suggesting reduced computational flexibility. This framework extends computational psychiatry by linking cognitive biases to neural dysfunction and provides a mechanistic approach to studying decision-making impairments in psychiatric disorders.

Substance craving and maladaptive choices are intertwined across addictive disorders. However, the computational mechanisms connecting craving and decision-making remain elusive. Here, we tested a hypothesis that momentary craving and value-based decision-making influence each other during substance-related reinforcement learning. We measured momentary craving as two groups of human participants (alcohol drinkers and cannabis users; total n=132) performed a reinforcement learning task in which they received group-specific addictive cue or monetary rewards. Using computational modeling, we found that, across both groups, momentary craving biased learning rate related to substance-associated prediction errors (RPEs), but not monetary RPEs. Additionally, expected values and RPEs jointly influenced elicited craving across reward types and participant groups. Alcohol and cannabis users also differed in the extent to which their craving and decision-making influenced each other, suggesting important computational divergence between the two groups. Finally, regressions incorporating model-derived parameters best predicted substance use severity in the alcohol, but not cannabis group, supporting the utility of using these model-based parameters in making clinical predictions for selective substance groups. Together, these findings provide a computational mechanism for the interaction between substance craving and maladaptive choices that is generalizable across addictive domains.

Reinforcement learning is a core facet of motivation and alterations have been associated with various mental disorders. To build better models of individual learning, repeated measurement of value-based decision-making is crucial. However, the focus on lab-based assessment of reward learning has limited the number of measurements and the test-retest reliability of many decision-related parameters is therefore unknown. Here, we developed an open-source cross-platform application Influenca that provides a novel reward learning task complemented by ecological momentary assessment (EMA) for repeated assessment over weeks. In this task, players have to identify the most effective medication by selecting the best option after integrating offered points with changing probabilities (according to random Gaussian walks). Participants can complete up to 31 levels with 150 trials each. To encourage replay on their preferred device, in-game screens provide feedback on the progress. Using an initial validation sample of 127 players (2904 runs), we found that reinforcement learning parameters such as the learning rate and reward sensitivity show low to medium intra-class correlations (ICC: 0.22-0.52), indicating substantial within- and between-subject variance. Notably, state items showed comparable ICCs as reinforcement learning parameters. To conclude, our innovative and openly customizable app framework provides a gamified task that optimizes repeated assessments of reward learning to better quantify intra- and inter-individual differences in value-based decision-making over time.

Grief is a reaction to loss that is observed across human cultures and even in other species. While the particular expressions of grief vary significantly, universal aspects include experiences of emotional pain and frequent remembering of what was lost. Despite its prevalence, and its obvious nature, considering grief from a normative perspective is puzzling: Why do we grieve? Why is it painful? And why is it sometimes prolonged enough to be clinically impairing? Using the framework of reinforcement learning with memory replay, we offer answers to these questions and suggest, counter-intuitively, that grief may have normative value with respect to reward maximization.

BackgroundCognitive deficits are a common symptom of depression and contribute significantly to the disabling effects of the disorder. Experimentally, they are observed as increased reaction times, increased error rates, and deficient performance adaptation after making errors or receiving adverse feedback, in multiple cognitive paradigms. In the current theoretical study we aimed to address the cause of these cognitive deficits. MethodsWe constructed computational models of optimal resource allocation in two cognitive tasks - Delayed Match to Sample (DMS), and Eriksen Flanker (EF). The models explicitly link performance feedback values and beliefs about task controllability with measures of cognitive performance including accuracy, reaction times, and post-error improvement in accuracy (PIA). We then introduced depression-related motivational changes - altered control belief and feedback values (representing learned helplessness, anhedonic valuation and negative bias) - to see if these factors can account for deficits in cognitive performance. ResultsIn the DMS task, altered control belief and lower valuation of correct performance accounted for decreased accuracy and decreased PIA. In the EF task, altered control belief and lower correct performance valuation could explain increased response times, decreased accuracy and decreased error-related negativity (ERN) signal. Increased valuation of adverse feedback, on the other hand, was linked to increased accuracy and the ERN signal. Furthermore, in the EF task, different combinations of depression-related motivational factors led to different patterns of cognitive performance, which could offer a basis for stratification. ConclusionsOur models offer an explicit computational and algorithmic bridge between the known depression-related motivation factors (learned helplessness, anhedonic valuation) and commonly observed cognitive deficits (increased reaction times, decreased performance accuracy, worse post-error adaptation), which contributes towards a better understanding of depression.

Pavlovian "approach or avoid" impulses are critical behavioral biases that, in excess, are linked to multiple psychiatric conditions. To investigate how such biases contribute to suicidal thoughts and behaviors, we analyzed data from two clinical populations completing an aversive Go/NoGo task. This task disentangles motor action (Go or NoGo) from outcome valence (escape from, or avoidance of, an aversive stimulus), enabling the isolation of Pavlovian biases from instrumental learning processes. We compared multiple computational models that had previously been proposed to explain Pavlovian tendencies, including reinforcement learning, active inference, and drift diffusion-based approaches. We employed a hierarchical Bayesian inference procedure that treats model identity as a random factor at the individual level, allowing an unbiased determination of which mechanisms most accurately captured participants behavior. Across both datasets, models featuring Pavlovian context biases plus a value-decay mechanism best accounted for performance. By contrast, policy-based Pavlovian models and more complex approaches, such as those integrating working memory or active inference, were supported by fewer study participants. These findings suggest that reflexive biases exert a persistent influence on decision-making, and that value decay plays a critical role in shaping behavior over time. Our results demonstrate the importance of systematically comparing and accounting for relevant cognitive processes to explain observed task behaviors. Understanding the factors contributing to task performance may help clarify how Pavlovian tendencies relate to psychopathology, including, in our case, elevated suicide risk. Finally, we illustrate how a complete hierarchical model selection framework can be applied to identify the most plausible mechanisms underlying Pavlovian biases, offering a robust approach for advancing our understanding of task behaviors and establishing clinical utility in future studies. Author summaryAutomatic "approach or avoid" reactions shape behavior, particularly in stressful or negative situations. In this study, we explored how these reflex-like tendencies might contribute to suicidal thoughts and behaviors. Two clinical groups completed a computerized task measuring responses to unpleasant sounds. Participants made either active responses (pressing a button to stop a sound) or passive responses (refraining from pressing to avoid starting a sound), allowing us to examine the interplay of automatic impulses and learning from past experiences. Our analysis showed that behavior was best explained by a model combining stable "approach or avoid" impulses with a forgetting process that reduced reliance on past experiences over time. More complex models involving strategies or memory-based control were less effective. These findings suggest that individuals with suicidal tendencies may rely on persistent reflex-like behaviors and over-index recent outcomes, compromising their ability to learn in uncertain environmental conditions. Understanding these cognitive processes provides insights into why some individuals feel trapped in harmful patterns of thought and behavior. Our work highlights how identifying shared traits in clinical populations using model-based methods can inform targeted mental health interventions and improve our understanding of cognitive functioning across disorders.

Difficulties in adapting learning to meet the challenges of uncertain and changing environments are widely thought to play a central role in internalizing psychopathology, including anxiety and depression. This view stems from findings linking trait anxiety and transdiagnostic internalizing symptoms to learning impairments in laboratory tasks often used as proxies for real-world behavioral flexibility. These tasks typically require learners to adjust learning rates dynamically in response to uncertainty, for instance, increasing learning from prediction errors in volatile environments. However, prior studies have produced inconsistent and sometimes contradictory findings regarding the nature and extent of learning impairments in populations with internalizing disorders. To address this, we conducted eight experiments (N = 820) using predictive inference and reversal learning tasks, and applied a bi-factor analysis to capture internalizing symptom variance shared across and differentiated between anxiety and depression. While we observed robust evidence for adaptive learning-rate modulation across participants, we found no convincing evidence of a systematic relationship between internalizing symptoms and either learning rates or task performance. These findings challenge prominent claims that learning difficulties are a hallmark feature of internalizing psychopathology and suggest that the relationship between these traits and adaptive behavior under uncertainty may be more subtle than previously thought.

Human instrumental learning is driven by a history of outcome success and failure. We demonstrate that week-long treatment with a serotonergic antidepressant modulates a valence-dependent asymmetry in learning from reinforcement. In particular, we show that prolonged boosting of central serotonin reduces reward learning, and enhances punishment learning. This treatment induced learning asymmetry can result in lowered positive and enhanced negative expectations. A consequential effect is more rewarding, and less disappointing, experiences and this may, in part, explain the slow temporal evolution of serotonins well-established antidepressant effects.

Currently, placebo-controlled clinical trials report mean change and effect sizes, which masks information about heterogeneity of treatment effects (HTE). Here, we present a method to estimate HTE and evaluate the null hypothesis (H0) that a drug has equal benefit for all participants (HTE=0). We developed measure termed estimated heterogeneity of treatment effect or eHTE, which estimates variability in drug response by comparing distributions between study arms. This approach was tested across numerous large placebo-controlled clinical trials. In contrast with variance-based methods which have not identified heterogeneity in psychiatric trials, reproducible instances of treatment heterogeneity were found. For example, heterogeneous response was found in a trial of venlafaxine for depression (peHTE=0.034), and two trials of dasotraline for binge eating disorder (Phase 2, peHTE=0.002; Phase 3, 4mg peHTE=0.011; Phase 3, 6mg peHTE=0.003). Significant response heterogeneity was detected in other datasets as well, often despite no difference in variance between placebo and drug arms. The implications of eHTE as a clinical trial outcomes independent from central tendency of the group is considered and the important of the eHTE method and results for drug developers, providers, and patients is discussed.

Substance Use Disorders (SUD) can be conceptualized as a prospective link from cues to craving and use. To explore the nonlinear relationships between craving and cues, this study applied dynamical systems theory (DST) to ecological momentary assessment (EMA) data. Optimized linear Seasonal Auto-Regressive Integrated Moving Average with eXogenous variable (SARIMAX) models were used to phenotype patients with SUD (alcohol, tobacco, cannabis, opiates, and cocaine), considering the potential for complex interactions between cue exposure and craving intensity in daily life. These phenotypic profiles were replicated in computational DST models to analyze the nonlinear interactions between cues, craving, and use. The study involved 211 individuals and 8,260 observations, with 154 patients fitting the SARIMAX model for the influence of cues on craving, and 57 patients fitting the SARIMAX model for a possible influence of craving on cues. Two DST models were adjusted to replicate the complex temporal dynamics of SUD based on these two directions of influence. The first DST model (adjusted to the influence of cues on craving) showed that an increase in cues leads to a rise in craving, which then diminishes both cues and craving itself, with use patterns following cravings trajectory. This patient profile is driven by a phenomenon of "maximum cue saturation". The second DST model (adjusted to the influence of craving on cues) demonstrated that an increase in craving was followed by an increase in cue reporting, leading to use, with use peaking and then reducing craving. This patient profile is characterized by a phenomenon of "maximum use saturation". Both models highlight craving as an essential modulator between cues and use, opening new therapeutic avenues.

Cognitive control deficits are consistently identified in individuals with schizophrenia and other psychotic psychopathologies. In this analysis, we delineated proactive and reactive control deficits in psychotic psychopathology via hierarchical Drift Diffusion Modeling (hDDM). People with psychosis (PwP; N=123), their first-degree relatives (N=79), and controls (N=51) completed the Dot Pattern Expectancy task, which allows differentiation between proactive and reactive control. PwP demonstrated slower drift rates on proactive control trials suggesting less efficient use of cue information for proactive control. They also showed longer non-decision times than controls on infrequent stimuli sequences suggesting slower perceptual processing. An explainable machine learning analysis indicated that the hDDM parameters were able to differentiate between the groups better than conventional measures. Through DDM, we found that cognitive control deficits in psychosis are characterized by slower motor/perceptual time and slower evidence-integration primarily in proactive control.

The brain combines sensory information and prior information, taking into account uncertainty, to perceive the world. This inference process approaches optimality in humans but with inter-individual differences associated with psychological traits. Previous results on these differences are in fact highly heterogeneous and even contradictory. We highlight experimental, modeling, and analysis choices that may contribute to this heterogeneity. We propose a set of tasks utilizing explicit and implicit priors, combined with computational modeling, to isolate the decision and learning stages of perceptual inference. Using a multidimensional approach, we characterized differences in perceptual inference associated with anxious, autistic and psychotic traits in two large samples from the general population. Our findings reveal that anxious, autistic, and psychotic traits form three distinct, yet correlated, dimensions. More anxious traits were associated with enhanced performance and greater reliance on sensory information at the decision stage. Autistic traits were not associated with any difference in perceptual inference; results for psychotic traits were inconsistent across the two samples. Results are partly different when using unidimensional analyses. Together, these results stress the importance of a multidimensional approach that takes anxious traits into account to characterize inter-individual differences in perceptual inference.

Inferring the true cause of noise--distinguishing between volatility (environmental change) and stochasticity (outcome randomness)--is essential for learning in noisy environments. While most studies rely on binary outcomes, previous models are designed for continuous outcome and use ad hoc approximations to handle binary data, introducing theoretical inconsistencies and interpretational issues. Here, we develop a normative framework for inferring the causes of noise from binary feedback that remains faithful to the discrete nature of the generative process and underlying statistical structure. First, we establish a generative model using a state space approach tailored for binary outcomes and derive the corresponding hidden Markov model inference procedure. Second, we introduce a computational model combining the hidden Markov model with particle filtering to simultaneously infer volatility and stochasticity from binary outcomes. Third, we validate predictions through a 2x2 probabilistic reversal learning task with human participants, systematically manipulating both noise parameters. Results show that participants adjust their learning rates consistent with model predictions, increasing learning rates under volatile conditions and decreasing them under high stochasticity. Our theoretical and experimental results offer a principled approach for dissociating volatility and stochasticity from binary outcomes, providing insights into learning processes relevant to typical cognition and psychiatric conditions.

Contingency Management (CM) is a psychological treatment that aims to change behavior with financial incentives. In substance use disorders (SUDs), deployment of CM has been enriched by longstanding discussions around the cost-effectiveness of prized-based and voucher-based approaches. In prize-based CM, participants earn draws to win prizes, including small incentives to reduce costs, and the number of draws escalates depending on the duration of maintenance of abstinence. In voucher-based CM, participants receive a predetermined voucher amount based on specific substance test results. While both types have enhanced treatment outcomes, there is room for improvement in their cost-effectiveness: the voucher-based system requires enduring financial investment; the prize-based system might sacrifice efficacy. Previous work in computational psychiatry of SUDs typically employs frameworks wherein participants make decisions to maximize their expected compensation. In contrast, we developed new frameworks that clinical decision-makers choose actions, CM structures, to reinforce the substance abstinence behavior of participants. We consider the choice of the voucher or prize to be a sequential decision, where there are two pivotal parameters: the prize probability for each draw and the escalation rule determining the number of draws. Recent advancements in Reinforcement Learning, more specifically, in off-policy evaluation, afforded techniques to estimate outcomes for different CM decision scenarios from observed clinical trial data. We searched CM schemas that maximized treatment outcomes with budget constraints. Using this framework, we analyzed data from the Clinical Trials Network to construct unbiased estimators on the effects of new CM schemas. Our results indicated that the optimal CM schema would be to strengthen reinforcement rapidly in the middle of the treatment course. Our estimated optimal CM policy improved treatment outcomes by 32% while maintaining costs. Our methods and results have broad applications in future clinical trial planning and translational investigations on the neurobiological basis of SUDs.

Whether we are making life-or-death decisions or thinking about the best way to phrase an email, counterfactual emotions including regret and disappointment play an ever-present role in how we make decisions. Functional theories of counterfactual thinking suggest that the experience and future expectation of counterfactual emotions should promote goal-oriented behavioral change. Although many studies find empirical support for such functional theories, the generative cognitive mechanisms through which counterfactual thinking facilitates changes in behavior are underexplored. Here, we develop generative models of risky decision-making that extend regret and disappointment theory to experience-based tasks, which we use to examine how people incorporate counterfactual information into their decisions across time. Further, we use computer-vision to detect positive and negative affect (valence) intensity from participants faces in response to feedback, which we use to explore how experienced emotion may correspond to cognitive mechanisms of learning, outcome valuation, or exploration/exploitation--any of which could result in functional changes in behavior. Using hierarchical Bayesian modeling and Bayesian model comparison methods, we found that a model assuming: (1) people learn to explicitly represent and subjectively weight counterfactual outcomes with increasing experience, and (2) people update their counterfactual expectations more rapidly as they experience increasingly intense negative affect best characterized empirical data. Our findings support functional accounts of regret and disappointment and demonstrate the potential for generative modeling and model-based facial expression analysis to enhance our understanding of cognition-emotion interactions.

Irritability and temper outbursts are among the most common reasons youth are referred for psychiatric assessment and care. Identifying clinical variables (e.g. momentary anxiety) that precede the onset of temper outbursts would provide valuable clinical utility. Here, we provide the rationale for a study to test the performance of classifiers trained to predict temper outbursts in a group of clinically-referred youth, in a home setting, enriched for symptoms of irritability and temper outbursts. Using observational data--digital based event sampling from previous Ecological Momentary Assessment data, we demonstrate promising results in our ability to predict the presence of a temper outburst based on clinical responses (e.g., whether the participant is grouchy, hungry, happy, sad, anxious, tired, etc.) prior to the emotional event, as well as external features (e.g., time of day, day of week). In exploratory analyses of existing data, consisting of n=57 subjects with a total of 1296 time points, we evaluate the feasibility of using a logistic regression-based classifier and a random-forest based classifier for predicting the temper outburst prospectively. In order to more rigorously assess these classifiers, we propose the collection of a large confirmatory set, consisting of at least an additional 20 subjects with an expected total of 400 time points, and will perform confirmatory analyses of the precision and recall of several classifiers for predicting temper outbursts. This work provides the foundation for the identification of features predictive of risk and future development of novel mobile-device-based interventions in youth affected with severe and impairing psychopathology.